Graphcut-based Interactive Segmentation using Colour and Depth cues

Segmentation of novel or dynamic objects in a scene, often referred to as background subtraction or foreground segmentation, is critical for robust high level computer vision applications such as object tracking, object classification and recognition. However, automatic realtime segmentation for robotics still poses challenges including global illumination changes, shadows, inter-reflections, colour similarity of foreground to background, and cluttered backgrounds. This paper introduces depth cues provided by structure from motion (SFM) for interactive segmentation to alleviate some of these challenges. In this paper, two prevailing interactive segmentation algorithms are compared; Lazysnapping [Li et al., 2004] and Grabcut [Rother et al., 2004], both based on graphcut optimisation [Boykov and Jolly, 2001]. The algorithms are extended to include depth cues rather than colour only as in the original papers. Results show interactive segmentation based on colour and depth cues enhances the performance of segmentation with a lower error with respect to ground truth.